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RELATED APPLICATION 

This application claims priority from provisional Application No. 60/215,762 
filed on June 30, 2000, the contents of which are incorporated herein. 

This application is related by subject matter to Application No. (atty. dkt. 



no. 3826-3) entitled "User Presence System and Method", fil ed concurrent! 
Ihe^contents Ct whtch'areincoTporatea nerem. 

TECHNICAL FIELD 

The present invention relates to a system and method for dynamic knowledge 
construction. 

BACKGROUND OF THE INVENTION 



The development of Artificial Intelligence (AI) modeled human understanding of 
the real world (RW), as summarized by Dreyfus et al., "Why Computers May Never 

20 Think Like People," Harvard Technology Review, 42 (Jan. 1986), is a collection of facts, 
rules of operation, and experiences. This collection of symbolic information varies with 
human skill in a particular application domain, by varying across a skill level scale from 
novice, through advanced beginner, and on to competent, proficient, and expert abilities. 
The variation of skill proceeds from the collection of facts to goal based action, with 

25 intuition and analysis resulting into the instant understanding of the expert. There is a 
response time associated with skill. Feigenbaum et al., "Signal-To-Symbol 
Transformation: HASP/SIAP Case Study," AI Magazine, 23 (Spring 1982) referred to the 
collection of RW data as a signal detection process. The construction of this data into 
symbols creates a modeled symbol-structure, composed of low-level attribute elements 

30 being clustered and grouped by RW objects (RWO) into an application hierarchy. 

This process of construction was called an inference between knowledge sources 
and observations using an IF-THEN hypothesis-testing algorithm. Inferencing could use 
pattern invoked, heuristic methods or logical, tree-searched, rule-based methods. Here, 
the connection between RWO and the modeled understanding is through an atomic object 



layer (with at least one attribute or feature) and a detection process on the existence of 
that attribute, with a probability of detection (Pd) and a probability of false-alarm 
detection (Pfa)- These atomic elements are built into more complex structures of an 
object diagram (OD). Solinsky, "An Artificial Intelligence Perspective on the Sonar 

5 Problem - Recognition Control Strategy in a Relational Data Base," #T85-1 199/301, 
Rockwell International, Anaheim, CA (Oct. 1985) showed that a relational database 
(RDB) was a means of constructing this knowledge base (KB) as that of clustering 
detected feature values of modeled RWOs. Here the example RWO is a tuple, and the 
fields of the RDB are the populated feature values. 

10 An important element of RDB access in this application is shown by Li, A 

PROLOG Database System , Research Studies Press Ltd., John Wiley, NY, NY (1984) to 
incorporate predicate calculus. Feature detection is the process of signal detection and 
feature extraction through algorithmic processing of atomic level sensor measurements 
using "a priori" knowledge, including a detection threshold setting value (d). Just as in 

15 the human skill model, an observation time (T) is required before a detection event can 
occur, and hence the symbolic construction process is discretized in time. Solinsky, "A 
Man/Machine Performance Model for Analyzing Sonar System Design", #T86, Rockwell 
International, Anaheim, CA (Dec. 1986) showed that the information rate of this 
symbolic construction process is constant, but varies in resolution analysis between 

20 sorting Yes/No decisions at low resolution, and analysis at high resolution, and involves 
feedback over a fixed construction/extraction time using short term and long term 
memory models (STM, LTM). After symbolic construction, a sequence of atomic events 
can be used to retain temporal or spatial event clustering (as in object spatial movement 
or eye-scanning motion). 

25 The false alarm rate (FAR) in feature-detection is the ratio of Pfa to T. The 

process of matching a set of detected object features to a new set of features is that of 
classification, and Solinsky, "The Use of Expert Systems in Machine Vision 
Recognition," VISION' 86 Conference, Detroit, MI, 4-139 (June 1986) showed that the 
Expert System (ES) paradigm was applicable to this process in vision applications. The 

30 feedback of the setting of the threshold detection value "d" is provided by the speed of 
the clustering processes required. This method of threshold controlled decision process 
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performance is based on one of five methods described later by Klir, Advances in 
Computers , Vol. 36 edited by M. C. Yovits, Academic Press, NY, NY, 255, (1993). 

The use of object detection in visual tasks is a human expert skill implemented as 
a Gestalt process. Solinsky has generalized the approach for a) feature selection (see 
5 Solinsky, "A Generalized Feature Extraction Approach," VISION '87 Conference, 
Detroit, MI, 8-57, June 1987), b) classification using neural -networks (see Solinsky, 
"Machine Vision Tutored Learning Using Artificial Neural Systems Classification," 
VISION '88 Conference, Detroit, MI, 1-MS88-490, June 1988), and c) decision 
processes for low FAR regimes (see Solinsky, "Evaluating System Performance in Low 

10 False Alarm Rate Regimes," #JCS/ASD/92-001, Advanced System Division, SAIC, La 
Jolla, CA, Feb. 1992). Solinsky, "A Method for Compact Information Characterization 
in a Finite, Discrete Data Set," #JCS/ASD/93-003, Advanced Systems Division, SAIC, 
La Jolla, CA (Apr. 1993) has also presented generalized information structuring for 
characterizing a discrete data set in a large dimensional feature space. This technique 

15 uses an important analysis method termed a "balloon matrix" for measuring correlation in 
N-dimensional hyperspaces. The hyperspace "diameter" of the balloon is found in a 
search time, which is related to the correlation lag and the correlation order. The 
compact form of a subspace representation uses a fractal dimensional ordering. In this 
way a much larger, and even infinite space can be made into a compact, set of finite 

20 spaces. Solinsky, et al, "Higher-order Statistical Application in Acoustics with Reference 
to Nonlinearities in Chaos," Third International Symposium on Signal Processing 
Apphcations (HOSSPA 92), Gold Coast, Queensland, Australia (Aug. 1992) has shown 
higher order statistical (HOS) correlation to exist in acoustic data from mammals and in 
neural data from bat brain neurons. Solinsky, et. al, "Signal Analysis of Nonlinear 

25 Dynamics and Higher-Order Statistics," SPIE Proceed. 2037 Chaos/Dynamics, San 
Diego, CA 163 (July 1993) has shown this HOS correlation to be present in other data set 
applications including financial data. This HOS correlation represents the Gestalt process 
of the N-dimensional hyperspace correlation (for N > 2). 

The structure for knowledge representation in Al has been dominated by the 

30 object model, with linked object diagrams (OD), such as shown by Booch, Object- 
Oriented Analysis and Design , Benjamin/Cummings Pub. Co., Redwood City, CA 
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(1994) with class attribute relationships, Coad and Yourdon, Object-Oriented Design , 
Prentice Hall Inc., Englewood Cliffs, NJ (1991) for super/sub object and associated 
object relationships, and the current modeling using the Universal Modeling Language 
(UML) by Fowler and Scott, UML Distilled , CRC Press, Addison Wesley Langman, Inc., 
5 Menlo Park, CA (2000), Runabaugh, Jacobson and Booch, The UML Reference Manual , 
Addison-Wesley Langman, Inc., Menlo Park, CA. (1999), and Booch, Rumbaugh, and 
Jacobson, The UML User Guide , Addison Wesley Langman, Inc., Menlo Park, CA 
(1999). 

The construction of information systems that are intelligent, by being self- 
10 erecting, and capable of implementation in a computing machine, was explored by 
Solinsky, "Intelligent Information Systems," SAIC White Paper, (Feb. 1995). Here the 
use of human interaction is used to construct information from data, as defined as 
"assessment by methods unrelated to the data itself." Intelligent systems have the ability 
to acquire and use information. These systems evolve information from data cases using 
15 an information "channel" of flow similar to the previously modeled information rate 
shown earlier by Solinsky (Dec. 1986) to be limited. The process in the intelligent 
information system using computing machinery involves a set of metrics: 

1) The information has a finite lifetime (x) and channel length L, which because of a 
fixed flow rate, c, limits the ability of the system to achieve instantaneous 

20 synchronization. Rather, a unit of synchronicity is finite, and is in terms of a 

"chron" (cm = cx/L). 

2) The distribution of information through replication must have a user's transaction 
cost (Cr) to limit resource wasting and time consuming processes. 

3) The system must incorporate a feedback process to minimize the average cm unit 

25 in the processes (cm = {cmlmin). 

4) The system customizes the user interaction to minimize the user's emotional level 
as determined through an interface (e.g., Gelemter, "The Muse in the Machine," 
Free Press Division of MacMillian, Inc., NY, NY (1994)). 

5) Diverse user interaction creates information latency, which incurs a cost (CO that 
30 randomizes information through equivocation (Solinsky (Dec. 1986)). 
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6) Diverse user interaction also creates information uncertainty as an injection of 
information randomness or noise (Solinsky (Dec. 1986)). 

7) User feedback is used to retain information accuracy by consistent and frequent 
decision confirmation. 

5 8) Both object-modeled information flow (e.g., OD) and data flow must exist in the 

system, where the information rate channel capacity (CC) limits the information 
flow rate, and the computing hardware channel bandwidth (BW) hmits the data 
rate. This process involves a user decision of transforming unknown data into 
objects for investigation and analysis (e.g., unfamiliar objects are data flow). 

10 Known objects are discarded by the user (e.g., familiar objects are information 

flow). 

9) The user actions are modeled by two levels of memory, STM and LTMs with the 
LTM model of familiar objects being accessed through a set of multileveled CC 
limited pathways. This provides an implementation of situation awareness for a 

15 specific user interaction. 

10) The system incorporates an overall cost function (CF) which limits the use of 
information flow by a redundancy factor (Re for Re = 1.0 as no information flow, 
and Re = 0.0 as an infinite flow for a state of confusion). This CF is a nonlinear, 
direct mapping of Re, and is inversely mapped to the information rate CC. The 

20 CF is user and appUcation dependent. The process of transforming data-to- 

objects is modeled as human perception and transforming objects-to-data is 
modeled as human instantiation. 

A series of research proposals (Murphy and Solinsky, "Automated Model 
Correlator and Metamodel Building Environments," Accord Solutions SBIR Proposal 

25 A95-065 (Jan. 1995), "The Information Computer - An IntelUgent Systems Component 
for Consistent Abstraction of Collaborator Experience," Accord Solutions (1996), and 
Accord Solutions ATP proposal "Components for a Concurrent Paradigm," (May 1997)) 
addressed the application of this information construction structure into a series of 
decision surfaces based on information content in the analyzed data. A series of briefings 

30 on the information system elements (Solinsky and Murphy, "The Information Computer", 
Accord Solutions briefing presented from 8/95-2/96 to McDonnell Douglas Corp. and 
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Cubic Corp.) identified an important concept in intelligent information construction 
systems, which can be implemented in computing machines. This information computer 
(IC) consists of: 

1) A LTM representation of information as a linked set of objects in an OD, with 
5 ordering based on the user's viewpoint, with objects of importance being closest, 

and objects of less importance being furthest away in "distance" (as a Unk count) 
from the most important object to the user. 

2) The OD viewpoint allows objects of self-containment to be chunked into a single, 
macro object, which has links to other objects, but is fully represented in the 

10 user's viewpoint context as a single macro object. In this way the entropy H, 

which is a function of the OD object count (O), link count (L), and average 

hyperspace spanning vector (D ) is defined as H = ^ (0,L,D ) and is minimized 
by chunking. In many instances, the chunked objects can capture a subspace of 
the general OD attribute set. 

15 3) The user decision/response is modeled as one of three outcomes of unfamiliar 

objects: a) discarded as not of interest, b) modified using STM to be corrected, 
and c) identified as being new, and entered into LTM. Familiar objects are 
automatically responded to with minimal time and effort as an automatic 
response. The use of a forced-choice interaction model with the user is a 

20 cognitive science method of creating confusion in the representation to force the 

user into a reactive and possibly emotional response which accesses the user's 
actual LTM information. 
4) The IC uses an Action Channel (AC) to construct confusion in the representation 
of the OD confronting the user. The AC is similar to a Shannon conununication 

25 channel for data flow, except the AC constricts the information flow as discussed 

earlier through equivocation and uncertainty. A specific element of the AC is an 
access to the OD stored in LTM, with the combination of new objects randomly 
included as input to the channel. The channel itself involves a type specified 
linking process (i.e., a verb) involving a propagation time step, which changes the 

30 output OD, by adding uncertainty through additional random links added and 
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removed from the OD as a noise process, and equivocation as an object removal 
process to be on the order of the input object count in order to retain the same 
entropy of input-to-output OD space. 
5) The AC model for OD modification is based on binary construction to a single 
5 object by a link type change as noun/verb/object/labeling for labels of: a) no 

change (NOP), b) create a new forward link (p addition), c) combine two objects 
(y combining), and d) create a backward link (b addition). The noun element in 
the change is the identity of the original object of the OD where the linking 
process occurs. 

10 The proposals by SoUnsky and Murphy extended the decision process to 

classification techniques using a neural network decision process and a generalized 
decision process as represented by Klir, Advances in Computers , Vol. 36 edited by M. C. 
Yovits, Academic Press, NY, NY, 255, (1993). The general decision process is an 
expansion of Pd/Pfa decision in a typical Ukelihood, cost function format, and includes a) 

15 classical set theory (Hartley, 'Transmission of Information", The Bell Systems Technical 
Journal L 535-563 (1928)); b) fuzzy set theory (Zadeh, "Fuzzy Sets", Information and 
Control 8 (3) 338-353 (1965)); c) probability theory (Shannon, "The Mathematical 
Theory of Communication," The Bell Systems Technical Journal, 27, 379-423, 623-656 
(1948)); d) possibility theory (Zadeh, "Fuzzy Sets as a Basis for a Theory of Possibility," 

20 Fuzzy Sets and Systems 1 (1), 3 - 28 (1978)); and e) evidence theory (Shafer, A 
Mathematical Theory of Evidence , Princeton University Press, Princeton, NJ (1976) and 
Demster, "Upper and Lower Probability Inferences Based on a Sample from a Finite 
Univariate Population," a) Biometrika 54, 515-528; b) Annals of Mathematical Statistics 
38, 325-339 (1967) as bounded probability, and behef functions (Shafer, "Belief 

25 Functions and Possibility Measures" in Analysis of Fuzzy Information Vol 1 edited by J. 
C. Bezedek, CRC Press, Boca Raton, FL, 51- 84 (1985))). Klir summarizes this decision 
process as being either involving fuzziness or ambiguity, where "strife-ambiguity" is a 
disagreement of alternatives and "nonspecified-ambiguity" is a set of unspecified 
alternatives, i.e. issues involved with decisions which require resolution because of 

30 disagreement or lack of information, e.g. at an emotional level. 

The decision process of the IC was modeled to include Klir's min/max 




uncertainty ranges of decisions to become invariant to uncertainty through the user 
decision process, such that the evolved LTM OD is the same information from all user 
viewpoints which then becomes a common set of information or knowledge. 

This background work in AI and information modeling has not included the 
5 combinations described for the IC and the use of neural-networks (NN) in decision 
processes of OD representations. Eliot, "Ruling Neural Networks," AI Expert, 8 (Feb. 
1995) has shown that a NN is not as easy to understand as an AI ES, which involves only 
rules. This is because ES logic involves discrete binary Yes/No states and NNs involve 
sigmoid-shaped decision surfaces with a focus on the "Maybe" region in the Yes-to-No 

10 transition region of the sigmoid. While ES's attempted to include this as a user 
input/review process with a certainty factor in decisions, this approach was unsuccessful 
because its final output was not a complete decision. The lure of the ES rule-based 
modeling is its compactness, but this can better be represented in a predicate calculus 
format with NN decisions as incorporated into this invention. The current invention 

15 transforms the OD models and AI of the IC to a mathematical hyperspace representation 
which is efficiently represented and operated on for RW applications using efficient bit- 
level manipulation and computation. 

A series of patents have dealt with ODs and the use of object models in 
applications, and particularly with RDB accessing. U.S. Patent No. 3,970,992 deals with 

20 a keyboard macro for retrieval application in a data processing system. U.S. Patent No. 
4,906,940 deals with a "rubberized" template matching approach for guiding an object on 
a road. U.S. Patent No. 5,506,580 involves a data compression approach using a 
character stream library encoding. U.S. Patent No. 5,548,755 is a hashing technique to 
optimize RDB-grouped query access. U.S. Patent No. 5,586,218 uses case-based 

25 reasoning for information gathering in a RW sensor derived data set. Here, decision and 
case construction use a Genetic Algorithm decision process. U.S. Patent No. 5,701,400 
constructs a tool-kit for financial advisors based on weighted logic ES, EF-THEN-ELSE 
rules to data sets in a RDB. 

U.S. Patent No. 5,712,960 incorporates abductive reasoning as a meta interpreter 

30 for updating a communication data base management system. U.S. Patent No. 5,768,586 
uses an object structure for data configuration in system modeling of complex 



enterprises, which begins with high level user descriptions and constructs low level 
descriptions from the modeled process. U.S. Patent No. 5,778,378 is a document retrieval 
application of an object-oriented (00) framework for word indexing and parsing. U.S. 
Patent Nos. 5,790,116; 5,794,001, and 5,900,870 use a GUI to construct a hierarchical 

5 definition of an object structure in a data record application. Templates are used for 
selecting data fields linkable to a collection folder of instantiations. Various 2-D graphics 
are used, such as a node-arc graph. U.S. Patent No. 5,806,075 uses a triggering 
methodology based on data values, of data duplications between local and remote sites. 
U.S. Patent No. 5,832,205 is a memory failure detection process based on comparative 

10 instruction analysis. U.S. Patent No. 5,875,108 is a GUI interface intelligence through 
adaptive pattern recognition of historic actions as in a button pushing effort using a VCR 
remote control. U.S. Patent No. 5,893,106 incorporates a server support to client users, 
which encapsulates a class hierarchy of 3-D graphics in data base appUcations. U.S. 
Patent No. 5,905,855 corrects errors in computer systems by two state analysis of initial 

15 and final state reference points. 

U.S. Patent No. 5,911,581 incorporates a metric for determining mental abiUty of 
complex task solving, and models reaction time, awareness thresholds, attention levels, 
information capacity and LTM access speed. U.S. Patent No. 5,915,252 uses a 
consistency check between a data source and target to simplify a user's job in data 

20 transferring. It includes protocol free construction of ES object links embodied in a 
common user interface. U.S. Patent No. 5,926,832 is a means of increasing memory 
access by memory address analysis and storage. U.S. Patent No. 5,936,860 is an 
application to warehouse control functions by a user with OO data modeling. U.S. Patent 
No. 5,953,707 uses a planned decision model in a client/server database from various 

25 user viewpoints in sales planning and inventory management as a user GUI without an 
00 model. U.S. Patent No. 5,958,061 uses a cache to store states for instruction 
translation. U.S. Patent No. 5,966,712 applies to the use of RDB storage of biomolecular 
sequences which, compares sequence frames and groups of frames and displays results to 
the user. 

30 U.S. Patent No. 5,970,482 applies to the application of feed-forward decisions to 

the data mining process. A predictive model is used to compare and rank symbolic data 



correlation significance. U.S. Patent No. 5,978,790 uses an edge-labeled tree approach to 
match input elements to output restructuring in a semi -structured database. The tree can 
only be structured in 2-D data base structures. U.S. Patent No. 5,991,776 involves an 
application of RDB indexing by linking tuple identifications to the document, but does 

5 not use an 00 structure. 

U.S. Patent No. 5,995,958 manages a database through the use of acyclic graphs 
by mapping an infinite data set to a finite storage of a X-function core representation. It is 
applied to RDBs w^ith user query support, but does not involve OO models, since the 
links are independent of the node contents. U.S. Patent No. 5,999,940 incorporates a 2-D 

10 visual representation for user discovery and visuaHzation for applications involving 
multiple access to databases, such as in health and doctor treatment areas. U.S. Patent 
No. 6,002,865 uses a multi-dimensional set of spreadsheet pages to construct a database 
by multiple levels of resolution. U.S. Patent No. 6,003,024 deals with row selection of 2- 
D database accesses for attribute-based record selection as an intersection of attribute 

15 correlation in 2-D. U.S. Patent No. 6,006,230 involves a client/server remote user access 
appUcation modeled in OO technology for proxy mapping. 

U.S. Patent No. 6,009,199 is a decision process for classification in decision trees. 
It is an iterative mapping of subspace to full space based on discriminative processes. 
The perceptron model is used in classification. 

20 The previously cited US Patent Nos. 5,832,205; 5,905,855; 5,926,832; and 

5,958,061 and U.S. Patent No. 6,011,908 deal with the iterative translation of computer 
microprocessor instructions to a target set of processor states embodied in a chip set with 
gated memory buffering. The process speeds up the instruction execution as historic 
references occur, and the system consequently becomes more adept at predicting the next 

25 sequential executable instructions. This is an N = 2 form of correlation prediction. 

SUMMARY OF THE INVENTION 

The system and method described in the detailed description below are usable to, 
among other things, generate outputs in response to real world stimulation. These 
30 outputs are generated by capturing concurrent inputs that are responsive to training 

stimulation, storing a model representing a synthesis of the captured inputs (that may be 



based on a mathematical projection of a hyperspace-modeled OD), and using the stored 
model to generate outputs in response to real-world stimulation. The system and method 
may be used in a wide variety of human interaction applications and apply to 
conventional storage and presentation devices, permitting the system and method to be 

5 easily incorporated into a variety of commercial products that utilize human-machine 
interfaces (e.g., HMI graphical user interface, or GUI) and current mass storage devices. 
By way of example, not limitation, the system and method may be applied to user 
presence and/or identification. In this case, the concurrent inputs may be handwriting 
and speech. In one particular example, the speech may be related to the handwriting such 

10 as when a person speaks his or her name while at the same time writing his or her name. 
A model representing a synthesis of these concurrent handwriting and speech inputs is 
stored. The model may, for example, be a worldline of linked object diagram exemplars 
in an N-dimensional space. The real world stimulation may be concurrent handwriting 
and speech inputs that are compared to the stored model, and the outputs (e.g., an identity 

15 verification) may be based on the results of the comparison. The use of a mass storage 
device, such as but not limited to an RDB, linked to the OD model by projections and 
NN/TC decisions in a binary taxonomy mapping, provides for a continuous expansion of 
each application complexity. The methodology of the present invention may be at least 
partly incorporated into a computer software program, a hardware processing engine, a 

20 specialized hardware application specific integrated circuit (ASIC) chip, or net Hst 
representations for a variety of ASIC technologies. 

The system and method may further include a forced choice interaction that 
generates one or more additional inputs that are captured and incorporated into the model. 
In the case of user identification, the forced choice interaction increases the probability of 

25 correct user identification, and can include measurements of user emotions and/or stress. 
For user identification, the forced choice interaction generates speech inputs that are 
responsive to a display of one or more images that may each include one or more letters, 
characters, numbers, symbols, images, etc. and combinations thereof. 

The present invention is described in the context of illustrative embodiments. A 

30 key concept of the invention is that the user's short-term and long-term memory is 
captured into an algorithmic framework by interaction with the user during "training" 



which then understands the decisive needs and wishes of each individual, and is able to 
dynamically map this information to low-level control functions used by commercial 
devices or to express this information to other users by using a common, intermediate 
method of information storage. This mapping uses a linked-object model in an infinitely 

5 expandable hyperspace (implemented as a bit-level computation), with synthesis creation 
based in user forced-choice input, and projected density derived, HOS feature 
comparison. By using a short and long-term storage approach, the utility of the concept 
is extendable to many applications with increasing complexity, with expandability only 
limited by the physical storage devices and communication hardware bandwidths. 

10 The complexity and expandability of user interaction can vary for each 

application, as does the variation and extent of the resources supporting this interaction. 
The simplest embodiment might be an apparent intelligent interaction response to the 
user with home appliances, which is through a single interface, far more advanced than a 
collection of remote control units (e.g. VCR remote controls, garage door opener, remote 

15 telephone extension, etc.). A numeric scale to represent this complexity (dubbed the 
Accord Scale, from 1 to 10 in logarithmic value; note that there is no value of "0" which 
would represent the complexity of N = 1, or a VCR control) is illustrated in Table 1. The 
OD entropy is a key element in the formulation of the Accord Scale complexity. 



Table 1 Accord Scale 



Numeric 
Value 


O.D. Space 
Dimension (N) 


Complexity 


Application 


1 


10 


Low 


Commercial electronic signature 
verification for single user. 


2 


50 


Moderate 


Internet music source search engine for 
single user finding of an MPS music 
selection of current tastes, independent of 
type, artist, musical instrument choice, 
etc. 


5 


100,000 


Medium 


Economic prediction models based on a 
number of users, cooperatively 
interacting with historic and globally 









dynamic databases, as developed from 
long-term memory. 


7 


500,000,000 


Very High 


Representation of the human genome 
code for a variety of health conditions 
compounded across multiple symptoms 
from many medical history examples 
with a utility in singular drug therapy. 



The specific examples are meant to show the growth in complexity of the user 
count, the historic and developed database encapsulation developed for each application, 
and the complexity of each output requirement. Note that the scale of N is not 
5 necessarily an OD volumetric growth, since the invention uses a more linear linking of 
the hyperspace. These examples are for illustrative purposes and a detailed discussion of 
electronic-signature verification is provided in the detailed description below. 

None of this prior art involves access in information structures for dimensions N 
> 3 or incorporates adaptive decision processes using neural-networks (NN) as a form of 

10 N-dimensional cell correlation representation. None of these approaches in information 
construction, organization, or prediction, incorporate higher-order correlation or HOS, 
and avoids the common AI ES, IF-THEN-ELSE, and logic-tree constructions. All 
template matching and correlation of this prior art are restricted to 2-D, linear techniques. 
None of this prior art incorporates an AC for user information testing with emotional 

15 sensing to construct common-information knowledge in a LTM, and also create 
confusion in a STM forced-choice decision process. None of this prior art incorporates 
the metrics of the IC in information flow parameters of rate c, channel capacity (CC), and 

information system length L and the temporal parameters of T5cm,cm, T, and cost 
functions Cr, Cl, CF and redundancy function Re. None of this prior art uses entropy, H, 

20 as a metric function of the OD representation vector space spanning distance D ^ for 
controlling analysis resolution, nor does it use predicate calculus for link location in a 
RDB. None of this prior art simplifies the OD model to a set of points in an N- 
dimensional space, which is expandable through an extension to the LTM storage of 



projection operators and decision surfaces, rather than explicit examples through feature 
attributes. By using a partially correlated axis space, which can be fractal in nature, this 
representation is expandable through subspace constructions at any later point in time, 
without loss of original model parameters. This invention addresses the original RDB 
5 constructions and avoids the approximations used in SQL development. The invention 
incorporates the simultaneousness of the input features in an N-D spatial correlation for a 
unique utility and storage of human and sensor information. 

The invention can be applied to human interaction applications that are not 
represented in N = 2 dimensional methods such as drawings, graphics, spreadsheets or 

10 database queries. These human interaction applications involve a complexity of 
dimensional correlation for very large N, which is beyond second order correlation and 
linear processes of N = 2 applications and instead the correlation is of a concurrent, 
nonlinear order N, as in Gestalt thinking. This concurrency is described through HOS 
representations and synthesis of the model. The system is a unique capabihty for 

15 information storage and retrieval which is beyond conventional graphic description for 
review, and is fundamentally user and hierarchically independent. 

These and many other advantages of the present invention will be more 
completely understood and appreciated by careful study of the following more detailed 
description of illustrative embodiments of the invention taken in conjunction with the 

20 accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows the invention application concept of user interaction through a 
25 simple device with other users, control functions, and an extension of the hand-held 
device to increasing resources embodied as a computer server device. 

Figure 2 shows the electronic-signature verification application example on the 
Accord Scale of complexity value 1, and involves a complexity in multi-channel user 
inputs for simultaneous correlation in electronic identification. 



Figure 3 details the invention system interfaces for a general application, with real 
world object sensing and control, and human user interfaces, with the fundamental 
mapping to a Core Model and a Mass Storage Device. 

Figure 4 details the Core Model elements of finite resource dimensions from input 
5 M' dimension and storage device M dimension, from the infinitely expandable N space of 
the application complexity. 

Figure 5 details a simple 2-D example of mapping an OD space of object 
locations and links to projected densities into lower dimensional subspaces which are 
characterized by density moments, here shown for N = 1. 
10 Figure 6 details an example of the subspace mapping expansion from N = 5, down 

to the simple 1-D densities of Figure 5. 

Figure 7 details the use of the invention technology derived from this invention in 
the electronic-signature verification application of Figure 2. 

Figure 8 details a flow chart of the invention method. 
15 Figure 9 details a generic block diagram for the invention system. 

GLOSSARY 

For ease of reference, the following glossary provides definitions for the various 
abbreviations and notations used in this patent application. 



AI 


Artificial intelligence 


HMI 


Human-Machine interface 


GUI 


Graphic user interface 


RW 


Real world 


RWO 


Real world objects 


Pd 


Probability of detection 


Pfa 


ProbabiHty of false alarm 


RDB 


Relational database 


SQL 


Structured query language 
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KB Knowledge base 

d Detection threshold setting value 

T Observation time 

STM Short-term memory 

5 LTM Long-term memory 

FAR False alarm rate 

ES Expert system 

HOS Higher-order statistics 

OD Object diagram 

10 UML Universal Modeling Language 

T Information lifetime 

L Information channel length 

c Information channel flow rate 

cm Temporal synchronicity unit or "chron" 

15 Cr Information replication cost function 

cm Average cm unit minimized by feedback 

Cl Information latency cost function 

CC Information rate channel capacity 

B W Hardware channel data rate bandwidth 

20 CF Cost function limits on information flow 

Re Redundancy factor 

IC Information computer 

H ICentropy of OD 

O IC object count in OD 



D IC OD hyperspace spanning distance vector 

L IC OD link count in OD 

) Function of contained elements 

AC IC action channel 

5 NOP No operation change in AC to OD for iteration cycle 

p AC labeling for forward link addition in AC to OD 

y AC labeling for combined link in AC to OD 

b AC labeling for backward link addition in AC to OD 

NN Neural-networks (form of decision mapping) 

10 00 Object-oriented 

M, M' Finite dimensional OD space for LTM mass storage (M) and RW 
interfaces (M') 

N Infinite dimensional OD space for STM decision processes 

n N-D space attribute axes 

15 n . i^^ attribute space axis 

i Location count index of n^ axis 

Xi,X2 2-D example of attribute axes 

d^ Location vector of o^^ index object exemplars in n attribute space 

0 Object index in OD space 

20 O Total count of o 

Pj Hyperspace 1-D profile projector onto the i^"^ axis of OD object 
locations 

Pi, P2 Histogram representation of P- projected profiles for axes xi, X2 




Hyperspace link vector between objects m and n of OD 

Hyperspace 1-D profile projector onto the i^*^ axis of OD link 
locations 

Histogram representation of projected profiles L- for axes xi, X2 
Class "c" distinction of subset of objects in vector o 

1-D index of link vectors 1^ 
Total count of index £ 

Subspace finite dimension count of OD from N space dimensions 
of OD 

Specific example set of dimensions in subspace count M (here 
used for N') 

Hyperspace M-D profile projectors for object and link densities to 
subspace m from OD space n > m 

Histogram representation of profile projectors for axes m 

Hyperspace M-D profile projectors for object and link densities to 
class-filtered object subset "c" in subspace M from OD space 
n and fn and c < O. 

Histogram representation of profile projectors , for axes m 

Central moments of 1-D population density of order m 
Total count of projected profile 1-D density 
Area Aq for M = 1 space projection 
Centroid mean 

Mean Ai for M = 1 space projection 

Centroid (area normalized) second central moment 



19 



Variance central moment of A2 for M=l space projection 

A3 Centroid (area normalized) third central moment 

m3 Skewness central moment of A3 for M = 1 space projection 

A4 Centroid (area normalized) fourth central moment 

5 m4 Kurtosis central moment of A4 for M = 1 space projection 

dc Simple class distance metric used in constructing class decision 

surfaces of second-order statistical densities (i.e. Gaussian). 

S Normalized m3 moment (i.e. S = 0 for Gaussian) 

K Normalized nu. moment with excess correction to Gaussian (i.e. K 

10 =0 for Gaussian) 

MV Machine vision 

SD Subspace diversity variable based on N and N' for each N' 

example 

pO^^) Image pixel density of subspace dimension m as a function of 

15 dimensional absolute distance x^. Used in computing moments 

and is a higher dimensional representation of Figure 3 histograms. 

A^ , A;^ , A^ , A3"' , A^ Extension of An, for m = 0,1,2,3,4 to m-D subspace 
central moment density weighted computations. 

20 S"™, K'" Extension of S and K to m-D subspace central moment density 

weighted computations 

TC Terminal cycle (form of decision mapping) 

S(i) Step operator value of random table lookup function from "pixel" 

value input seed at location address "i" in TC mapping 

25 S„ Location vector of addresses generated in n dimensional space 
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based on TC sequences using step operator function values S (i) for 
a stable TC "orbit" 

8 nm Eccentricity of major and minor axis in 2-D (n,m) representations 

LAN Local-area network 

5 GPS Global Positioning System 

RF Radio frequency 

ASIC Application-specific integrated circuit 

LPC Linear predictive coding 

ID Identification 
10 Y Count of TC iterations 



DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS 

15 The invention is described with reference to a typical example application as an 

intelligent interface between two classes of elements used in many product-based 
application devices as shown in the invention application concept of Figure 1. Figure 1 
shows a human user 100 interacting via a hand-held unit 102 with other users 104, as in a 
communication system, and with other devices 106, as in a television control system. 

20 This interaction is two-way: as in speaking into a microphone 108 of hand-held unit 102 
(e.g. "say"), or punching keys on the unit's keyboard 110, or moving in front of the unit's 
camera 112; and as in reading numbers displayed on the hand-held unit's display 114 
(e.g. "see"). The human user 100 is able to communicate with other users 104 and the 
controlled devices 106 through a tailored, dynamic interface that is based on the short- 

25 term (as in current and recent thoughts) and long-term (as in past and remembered 
thoughts) user's dynamic memory recollections. The hand-held unit 102 has a built-in 
memory (not shown in Figure 1) that stores the specific user(s) preferred interaction 
methods as a short-term memory, and has additional memory to extend this mapping via 
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a connection to a network server 116 for long-term memory storage. This extended 
mapping (networked database) contains information from multiple users, and in itself 
captures commonality ampng the user interactions. 

A specific example application of this invention concept is shown in Figure 2 for 

5 electronic signature verification and specific user presence confirmation, dubbed e- 
presence. Here, the user is using a simple, commercial input device, such as a Personal 
Digital Assistant (PDA) 10 with a stylus pressure keypad and a microphone input. The 
microphone 1 is used to acquire input data from the user 4 speaking one's name (as 
indicated by the line associated with reference numeral 5), at the same time that one is 

10 signing one's name by hand 6 on a signature pad 2 using a stylus 3. In familiar three- 
dimensional space, a person or object can be located by specifying three coordinates. In 
an X-, y-, z-space, the x-, y-, and z-coordinates are specified. In accordance with the 
present invention, additional axes are added to generate an N-dimensional space. For 
example, one additional axis may represent time, another additional axis may represent 

15 temperature, etc. The current x position, the current y position, and the current z position, 
the current time, the current temperature, etc. define a particular point within this N- 
dimensional space. The changes over time of x position, y position, z position, time, 
temperature, etc. define a path or a "worldline". The correlated speak/sign signature 
verification constructs unique user identification of attribute space dimension N = 10 (5 

20 signature features and 5 aural/spoken features) and consists of a linked OD in the 
invention technology for comparison to previous user "training" signatures. This first 
phase of the e-presence application is not just using signature recognition, as might be in 
a thumb print reader, and susceptible to forgery or facsimile input, nor is it just using 
aural identification of the user's voice, which could be compromised by using a tape- 

25 recorded spoken name. Rather, the e-presence system is capturing a dynamic event of a 
speak/sign correlation of motions and actions, which is expressed in all the manners of 
the specific user via the N = 10 dimensional space concurrency. The time sequence of 
this speak/sign process creates a "worldline" of linked OD exemplars in the ten- 
dimensional space. Of course, speak/sign are merely examples of concurrent inputs that 

30 are responsive to training stimulation and the scope of the invention is not limited in this 
respect. In fact, the invention could apply to an interface for a severely handicapped 
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person, whose biometric inputs might be just a flutter of eyelids in front of an eyepiece 
camera. Also, the invention is not limited to two concurrent inputs and is generally 
applicable to two or more concurrent inputs. The number and type of concurrent inputs 
will generally depend on the particular application developed in accordance with the 
5 inventive concepts described herein. 

It is expected that because of this high dimensionality, the first phase of the e- 
presence process of N = 10 will have a probability of correct (Pc) user ID after only a few 
training sessions (e.g. less than ten), and will be Pc > 95%. However, this is not at all 
sufficient for electronic signature applications in commerce, and so a second phase of e- 

10 presence involves the use of a sequenced display 7 of the PDA 10. Here, a set of letters 
(or numbers) are sequentially shown to the user to see (as indicated by the line associated 
with the reference numeral 8) each letter, one at a time, which, after each display, one is 
asked to say (as indicated by the line associated with the reference numeral 9) the 
displayed letter name. The rate of display is based on a forced-choice user mode (on the 

15 order of five per second), and includes the dynamics of the sequence. While a set of 
letters are sequentially shown in this example, the sequenced display 7 may generally be 
images that may each include one or more letters, characters, numbers, symbols, images, 
etc. and combinations thereof. 

Thus, the see/say part of the second phase includes a dynamic recognition of 

20 again an N = 10 attribute representation, where here all the attributes are based on the 
aural spoken sounds of a known-displayed letter. The sequenced exemplars create a 
second user worldhne. The Pc for this second phase event is expected to be Pc > 90%. 
Because the displayed letters are random and change each time the system is used, the 
false detection rate here is expected to be much lower, resulting in a combined first and 

25 second phase of e-presence to be a combined Pc > 98.5%. 

This example application may be further embellished by including increased 
complexity of failed first phase testing, and user dynamic profile updating as well as 
stress detection and emotional sensing overrides. In summary, the e-presence application 
precedes through the steps of Table 2. 
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Table 2: e-presence Application Steps 





Action 


N 


Pc 


Pc 


1 


Sign/Speak 


(5 aural; 5 signature) 


95% 


or < 95% (increase 
complexity) 


2 


Repeat 1 if Pc <95% 


(10 aural; 10 signature) 


98% 


or <98% (stop) 


3 


See/Say 


(10 aural) 


90% 


or <90% (increase 
complexity) 


4 


Repeat 3 if Pc< 90% 


(20 aural) 


90% 


or <90% (stop) 


5 


Accept Identification 









The invention is further detailed in this appHcation after the following initial fundamental 
description of the technology. 

5 Figure 3 shows the key elements of the application concept shown in Figure 1 and 

the example e-presence application shown in Figure 2. Figure 3 shows a means of 
interfacing a general purpose user and sensor/effector-based commercial device to a 
relational database (RDB) mass storage device which provides an isomorphic mapping of 
human knowledge interaction for dynamic intelligent responses that are independent of 

10 the appHcation domain (or multiple domains) or user (or multiple user) specifics. The 
advantages of this approach over the prior art are that the simplistic user and RW 
interfaces can create a concurrency of stimulations in a high dimensional space for more 
precise modeling, which is also storable using a simplistic 2-D or 3-D mass storage 
method. The first class of elements shown in the figure is for user and sensor interfaces. 

15 The user-receiving interface 11 includes data outputs in the form of visual displays in 
graphics or text, aural as sounds or speech, and kinesthetic as vibration or 
thermal/electrical stimulation. The user transmitting data inputs interface 11 includes 
data inputs in the form of kinesthetic-controlled device elements such as mice, keyboards, 
gloves, or joysticks; chemical/odor elements as in human odors or skin 

20 moisture/sweating; and spoken sounds or word elements. In addition, other RW sensor 
inputs 12 are used in the example interfaces, which can include, but are not limited to 
electronic camera/video data, GPS location sensing, remote thermal sensing of human 
elements in scanned or image data, and possibly switch based element motion. Finally, 
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RW control output devices 12, such as lever arms, motors, and robotics are used as 
effector devices. 

The second class of elements shown in Figure 3 and used in these same 
applications are for the mass data storage devices 13, which can be the computer server 
5 shown in Figure 1 and can be constructed of typical read/write functionality in magnetic, 
magneto-optic, random access memory (RAM), flash electronic memory, and any other 
electronic, or optical/electro-mechanical devices capable of storing and retrieving data 
values in digital format (bits, bytes, words), which are inherently in a physical structure 
and have an access format in two dimensions (as surface cells or surface locations) or in 

10 three dimensions (as volumetric cells such as holographic optical access or collections of 
surface locations, such as disk platters). These interface structural dimensions can be 
summarized as linking a LTM, physically contained in a low-dimensioned, physically 
constructed mass storage device 13 (i.e. with a finite dimension <3), to a RW access 
method involving human and sensor/control interactions of various attributes and also 

15 being of finite dimensions 11, 12. 

The system and method constructs three properties of algorithmic function, also 
shown in Figure 3, that include a Core Model 14, and two mechanisms for input and 
output mapping to the Core Model. The first mechanism is a projected atomic 
description 15, which maps elements of the Core Model outputs to the user interfaces 11 

20 and the RW effector interface 12, and a decision surface input construction 16 which 
receives data inputs from the user input interface 1 1 and the RW sensors input interface 
12 and uses an adaptive threshold, NN decision process. This decision surface 
construction combines these inputs into modification of the atomic descriptions of the 
Core Model. 

25 The Core Model 14 of Figure 3 includes elements shown in further detail in 

Figure 4. Figure 4 shows a means of creating an autonomous construction engine of an 
object space model for this isomorphic knowledge representation which contains N 
dimensionality of an OD beyond the atomic RW interface attributes, in order to 
synthesize the complexity of all types of encounters of the engine using non-orthogonal 

30 projections with a reduced set of M' RW features. This N-D space OD is represented as 
point-location populations in a mathematical representation for objects and object-links. 
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These elements include a RW interface to the two input/output elements of Figure 3 (15, 
16) as an OD atomic layer interface 17; and includes an OD-linked node representation 
18 in the format set of predicate calculus locators accessing the mass storage device 13 of 
Figure 3. Both the RW interface 17 and the locator interface 18 are constructed in a 

5 mathematical space of M and dimensions, where this dimension (M), in general, is 
finite. Each of these M-dimensional space interfaces is mapped into an N-dimensional 
structure representation that can be infinite and extendable from a finite space. This 
mapping is based on a nonlinear subspace mapping from M'-^N (engine 19) and M^N 
(engine 20) in either direction. The N dimensional space 21 contains the structure of an 

10 OD represented in mathematical form. The advantages of this approach over the prior art 
are that the N-D space constructs a more representative synthesis of the input data for 
modeled representation. 

The construction of the N-dimensional (N-D) object space represents the object 
attributes as axes in this space of vector n , where each vector value ni (for i = 1 to N) is 

15 the i^*^ attribute value of an object that spans the complete N-D dimension. Most, if not 
all, objects of the OD used in this space for representation consist of attributes, where i is 
a subset of the complete space, and hence exist in a smaller representational space or 
subspace. 

As a simplistic example, a 2-D form of an OD is shown in Figure 5. Figure 5 
20 shows a means of analyzing the projections of the synthesized OD representation in 
localized subspaces for efficient metric evaluation as a STM process in the automatic 
refinement of the OD representation. A set of objects 22, such as the script codes of area 
and length in the signature of Figure 2, with attributes graphically represented by a two 
axis scale, (Xj , X2), are represented as two classes (such as developed from two user 
25 training sessions) by crosses for the first set, and the second set of objects are represented 
by circles. These discrete sets are based on an object classification construction from 
previous training sessions or a priori information. These objects have location vectors d^ 

in this space and are point occupations, constructing a discretized set of exemplars similar 
to those used in NN techniques. The OD space is partially orthogonal, in order that a 

30 projection operator, P- can project a "profile" of the OD onto the i^^ axis as a density, and 




is shown in Figure 5 as a histogram count of the object density in discretized binning of 
the axis space. OD attribute-projected histograms are shown for both axes as Pj , . 
The more common 2-D representation of an OD includes a link between any two objects 
to show a relationship. In this N-D representation, the link is a second vector 1^^ shown 

5 in Figure 5. While the typical 2-D graphic OD has many links usually shown, for 
discussion purposes, only one link is shown in Figure 5. A second type of description, 
using a similar operator to the projection operator in F- object density profile histograms, 
is used to represent link density profile histograms as the occupation of the same 
discretized space with a link, projected by the link projection operator, Lj , and shown as 

10 a single example set Li, L2 for the Xj and X2 axis of the one link spanning two grid cells 
in Figure 5. Note that the link density histograms occupy discrete grid points that are on 
the half spacing to the object density grid point spacings. In the simplest representation, 
the links could be the worldline between the OD exemplars of the atomic features in the 
speak/sign application. The advantages of this approach over the prior art are that the 

15 projected atomic outputs from the LTM tuple-based RDB storage of sub-space operators 
have a higher fidelity in matching as RW outputs and user accepted representations. 
These simple mathematical concepts are extended to the general mathematical concept of 
the invention in the following summary. 

In summary, the N-D object space description consists of o'^ indexed objects 

20 (with a total count O, and optional class distinction C^) located with a location vector 

do, and ^'''indexed links (with a total count L^), indexed between m^*^ and n^^ objects as 

vector Projection operators operating from the complete N-D space to a single 1-D 

space of axis Xj created histograms of object density (P^ operator created histogram 

density Pi(Xj) and P2 operator created P2(x2)) and link density (L^ operator created 

25 histogram density Li(Xj ) and L2 created L2 (X2)). These operators can be generalized 
to constructions which map larger, N-D spaces to subspaces that are M-D, where the 
example shown in Figure 5 was for M = 1, and N = 2. Hence, the general projection 

operators project to an M-D space as P^ and L - ^ where m is the subspace dimensional 



set (indexed to a total of M count). Note that these dimensions are not necessarily 
sequential, as for example with M = 4, m = {1,3,5,9}, and for M'= 4, m = {1,2,6,8}. 
These generalized mapping functions are shown below for an OD without class 
distinction (C^ = 0), and a multi dimensional histogram density of m index. 

5 p. =P_ { d^j }n for all object classes and n > fn (1) 

Lfh = L^n { -^^ } n for all object classes and n > - m (2) 

with ^containing linked object pairs (m, n) from space fi , and and L^^^ have 

10 histogram density dimension in in . 

One can also form a class filtering operation using NN decision surfaces for the 
specific c'^ class of the full class set , where c is a subset index of 6 as: 

P^ ^ {do}n for c subset object classes (3) 
15 V^=V^{£^}^ for c subset object classes (4) 

Structure Recognition in N-D Space 

The concept formation of the invention constructs an N-Dimensioned space of 
discretized point occupancy, where the axis defines an object attribute, and the location 
20 on that axis defines the attribute value. The object attribute point location is contained in 
a set of object vectors dg. The objects can be contained in groups of similarity as 
classes, and can be filtered as class object populations from the complete space with 
subset . The relationships between objects are contained in a second set of vectors, as 

a vector between the m^^ and n^^ object, as link vector 1^^ . The naming of the objects is 
25 contained in the class-filtering operator . 

The full N-D space is expandable to infinite dimensionality and hence can contain 
sufficient entropy to handle any application. The "curse of dimensionality" of this 
entropy is not an issue, because the population of the space is inhomogeneous, and the 
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linking density is more linear, rather than volumetric, such as in worm-shaped occupation 
densities. The use of a balloon matrix (see Solinsky, "A Method for Compact 
Information Characterization in a Finite, Discrete Data Set," #JCS/ASD/93-003, 
Advanced Systems Division, SAIC, La Jolla, CA (Apr. 1993)) to order this structure by 
5 spanning radiuses is another metric of the finite population of the N-D space entropy. 
Any single axis of the space can be considered an atomic layer as mappable to the RW 
input/output functionality (11, 12), or to the mass storage device LTM memory cells (13). 
These M-spaced cells are mapped to the N < 3 storage device structure using predicate 
calculus locators. 

10 The invention includes a means of performing subspace mapping to dimension N' 

< N, and N'> M' for complexity representation which minimize the OD entropy H, and 

hyperspace spanning distance D as a function of (N, N^ M'). Because the use of 
subspace projection operators on objects and links is a form of compaction, these 
subspace histogram projections are observable discriminations for synthesizing the actual 

15 OD in its full dimensionality. The statistics of a histogram population density can be 
modeled from the simplest structure, to the most complex, by the use of the Characteristic 
Expansion Function (see Kendall and Stuart, The Advanced Theory of Statistics , Vol. I- 
III, MacMillan Pub. Co. (1997)), which is a construction of ordered Am moments of this 
population density (orders for m > 2 are HOS). 

20 The invention provides a means of utilizing HOS features in representing 

asymmetry for recognition metrics in the model synthesis which can be expanded to 
orders beyond 8^^ order, but are claimed operationally successful at 4^^ order. As an 
example, the first five moments for the M = 1 space of Figure 5 are: 

25 Ao = total count of projection density (area a) (5) 

Ai = centroid mean ( |li ) (6) 

A2 = centroid area normalized variance or second central moment ( a^) (7) 
A3 =centroid area normalized skewness or third central moment (ms) (8) 
A4 =centroid area normalized kurtosis or fourth central moment (nu) (9) 

30 and have been shown (see Solinsky et al, "Higher-order Statistical Application in 



29 



Acoustics with Reference to Nonlinearities in Chaos," Third International Symposium on 
Signal Processing Applications (HOSSPA 92), Gold Coast, Queensland, Australia (Aug. 
1992) and Solinsky et. al, "Signal Analysis of Nonlinear Dynamics and Higher-Order 
Statistics," SPIE Proceed. 2037 Chaos/Dynamics, San Diego, CA 163 (July 1993)) to be 
5 useful in well characterizing RW data. A cluster object or link density will eventually 
project to a Gaussian distribution density described by a centroid mean and variance 
because of approachment of the central limit theorem for large O or small M for large N. 
Two such class filtered projections, such as the crosses (user statistic #1) and the circles 
(user statistic #2) of Figure 5, can be discriminated by using a classification normalized 
10 distance dc defined as: 

de ^(^ir 1^2 fl<^^<^2 (10) 

where dc is used to locate decision surfaces in the OD, and where the indices of the 
15 moment are based on class-filtered projections (from Equations (3) and (4)). Distance 
classification techniques of the metric in Equation (10) can be expanded to complex 
formulations including NN forms (see Solinsky et al, "Neural-network Performance 
Assessment in Sonar Applications," IEEE Conference on Neural Nets in Ocean 
Engineering Applications, WDC,\ (Aug 1991)). 
20 Multimodal distributions of projected densities contain higher-ordered moments 

(e.g. A3, A4), and can be simplified using a Hermite ordering with coefficients of 
skewness (normalized ms as S = ms/a^), kurtosis (normalized m4 with "excess" Gaussian 
removal as K = (ni4/a'*)-3), and super/hyper skewness/kurtosis to 8^^ order (see Solinsky, 
"Trispectrum Utilization in Higher Order Statistical Applications," Proceedings of IEEE 
25 Conference on HOS, Grenoble, France (1991) - also in Higher Order Statistics , J. L. 
Lacoume Editor, Elsevier Science, Ltd, Netherlands, 339 (1992)). This cumulant 
statistical ordering with Hermite polynomials is the preferred HOS approach for 
efficiency. 

The identification of finer detailed structure beyond these moment representations 
30 can be modeled as hyper-spheroids with asymmetry, first as dumbbells of a single axis of 
spheroid coupling, and then as multiple axis dumbbells. The amount of asymmetry is 
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contained in the major/minor axis eccentricity Enm between axis indices n and m, and is a 
pair wise spatial set. An algebraic geometric modeling can be constructed to represent 
these structures, and through synthesis the projected densities can be represented as a 
geometric construction at a higher dimensionality by being projected in a manner to well 

5 match the RW population. Mean-root, summed error differences can be used as a simple 
synthesis feedback metric or "cost function" comparing the RW to the synthesized RW 
model. A critical component in the development of the structure formulation is contained 
in the use of Machine Vision (MV) techniques in analyzing these densities as images in 
2-D projections. Hence, the invention also utilizes a means of incorporating MV 

10 technologies in assessing the symmetry of the N' subspaces. This is an extension of the 
moment ordering shown previously, but represents structure by features, which can be 
generalized (see Solinsky, "A Generalized Feature Extraction Approach," VISION '87 
Conference, Detroit, MI, 8-57 (June 1987) and Solinsky, "A Generalized Image 
Enhancement for Machine Vision Architecture", Ultratech Vision West Conference, 

15 Long Beach, CA, 4-47 (Sept. 1986) and incorporate NN decision processes (see Solinsky, 
"Machine Vision Tutored Learning Using Artificial Neural Systems Classification," 
VISION '88 Conference, Detroit, MI, 1-MS88-490 (June 1988)) for feature 
detection/construction. It is through this analysis/synthesis process of the N-D 
representation that the core construction of the STM/LTM mapping can be accomplished 

20 with accuracy only limited by the dynamics of the iteration in the comparison metric. 

A critical element in the structure recognition is to utilize subspaces of reduced 
dimension, N', where N'«N, but N' > 3, such as N' =10. Here the projections are made 
to a set of "images" in reduced dimension. These images are made in the same manner 
that Figure 5 showed a 1-D "image" histogram "pixel" density (object/link) from N = 2, 

25 and "profiles" of these images to 1-D, and a 2-D image pixel density from the "complete" 
N = 3 space. However, the projections would be in the case of N' = 10, from an original 
larger dimension of N, to a set count diversity variable (SD) of (N- N') projections, with 
each expanded into a lower dimensional space, for a larger total SD number of "image 
pixel densities", p(x^)of (N-N') diversity in subspace dimension, m. 

30 The expansion of space density count to multiple subspaces of reduced dimension 



N'< N is shown in Figure 6. The original example space has a single dimension of N' = 
N = 5, with no diversity (SD = 1) as shown in Figure 6a. There are SD = 5, N' =4 
dimension subspace expansions 27, with different axis diversity as shown in Figure 6b. 
Each of these N = 4 spaces can be expanded 28 into an N' =3 subspace diversity, with 

5 total diversity SD = 5 x 4 or shown in Figure 6c. Finally, for N' = 2, the subspace 
diversity (SD = 5 x4x3) forms image spaces similar to Figure 5 as shown in the output 29 
of Figure 6d. The last subspace used in the HOS metric calculation of Equations (5) 
through (9) is shown in Figure 6e expansion 30, with the highest diversity (SD = 
5x4x3x2). Thus, in general, one can see that the subspace diversity of reduced 

10 dimensionahty operates in (N - N^-D space as: 



The entropy of this diversity can be modeled as an approximation to the functional 
definition in (O, L, D ) as: 



It can be shown that multi-dimensional moment density construction for 
dimension m, with m = (N -N") from these subspaces, will completely represent all image 
pixel constructions, using a weighted construction in the same manner from Equations (5) 
through (9) with object population pixel density p(x^) as: 



SD = (N!)/(N' 



'!)forN.>N' 



(11) 



H = SD Irz (SD) 



(12) 
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20 



p(x m ) d ^ m = normalization "area" of dimension m 



(13) 




(14) 



25 




(15) 




(16) 



A:= J (x.-Ar)'p(xJdx^/Ao"^ (17) 

and the similar Hermite polynomial coefficients S"" and K"" apply. This approach of the 
5 invention is a means of capturing the HOS correlation of OD attributes in N' dimensions, 
when N' > 3, with each HOS order corresponding to an increase in dimensionality for N' 
subspaces. 

Hence, a mathematical process using MV central moment density algorithms is 
described to construct a large feature set for quantifying multimodal, projected density 

10 asymmetries by subspace computation, and a synthesis feedback is used to refine the - 
postulated to real N-D space occupancy fidelity of any accuracy. This projected density 
of Figure 5, described by object and link density operators of Equations (1) and (2) are 
seen in Figure 5 as image "profile" pixel histograms, where the object pixels are counted 
at axes locations within grid cells, and the link pixels are counted at axes locations 

15 straddling grid cells, or at a Vi grid-cell offset. These histograms can be described by a 
feature set based on HOS moment expansion of Equations (5) through (9) for the 
simplistic low dimension (N = 2) pixel set of Figure 5, but can also be represented by the 
generalized HOS moment expansions of Equations (13) through (17) to fourth order, and 
are anticipated to not require representation beyond eighth order, but in principal can 

20 extend to all orders. This set of HOS becomes the feature set used for deciding upon the 
accuracy of the OD representation. This approach of the invention provides a means of 
continuously expanding the OD dimensionality to a dimension of the OD of N > N' 
where N can be infinite, based on a fidelity controlling parameter in the synthesis metric 
decision threshold d parameter. This advances prior art to unique levels of complex 

25 modeling. 

Thus, a feedback process is constructed within the space to synthesize a 
dimensional space of order N > M' to represent the observed ordering of the RW 
interface atomic dimension being limited by the interface observable counts of 
input/output devices in (11) and (12) of Figure 3. The.N representation space is 
30 expanded and populated with random data pixels, and then iterated and compared with 



the subspace projection using the HOS metrics. The expansion and projection synthesis 
does not have to be sequential as shown in Figure 6, nor does it have to use fully 
orthogonal (i.e. eigenvector decomposition based on Principal Component Analysis) 
projections. It can utilize partially projected axes orthogonality, as used by Independent 

5 Component Analysis techniques (see e.g. Sejnowski and Churchlang, The Computational 
Brain , MIT Press, Cambridge, MA (1992) and Lee, Independent Component Analysis- 
Theory and Applications , Klumer Academic Publication, Boston, MA (1998)). The HOS 
features and metric comparisons will still carry all of the discrimination. These 
projections can also be of fractal order to axes of partial correlation. In this manner, a 

10 completely ordered representation space is possible, and extendable to N =oo , with 
diversity in the subspace, SD, being quite large, yet finite (N - oo), no matter how 
complex the atomic layer becomes, where eventually, N' = M' satisfies the RW mapping 
dimensionahty using a variety of N' examples. The simplification of this synthesis 
construction is contained in a nonlinear and nonsequential search strategy, using the 

15 principles of the AC, that optimally discovers the required object- and link-pixel 
representation for the RW example data at hand. 

Neural-network (NN) Terminal Cycle (TC) Decision Surface Classification 

An element in simplifying the N-D space construction and in creating asymmetric 
20 projections is to use the class filter in the projection operator as described by Equations 
(3) and (4) and construct decision surfaces with NN technology. This increases the pixel 
complexity, from a Central Limit Theorem growth to second-order statistics with no 
asynraietry metric, to a more complex asymmetric projection. Class construction 
requires decision surface formation in the M-D space. 
25 Solinsky et al, "Neural-network Performance Assessment in Sonar Applications," 

IEEE Conference on Neural Nets in Ocean Engineering Applications, WDC, 1 (Aug 
1991) has shown the superiority of NN decision surfaces for class surface constructing 
over the simple distances formulations used in Equation (10) for bimodal separation. 
This superiority is due to the nonlinear surface construction over the linear form of 
30 Equation (10). NN decision surfaces are able to include surfaces of quadratic with one 
hidden layer, and hyper-quadratic with two hidden layers (see Lippman, "An Introduction 
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to Computing with Neural Nets," IEEE Acoustics, Speech and Signal Processing Mag. 4 
(2), 4-22 (April 1987)), implying that the N-D classification layers would be beyond two 
hidden layers in complexity. A second means of decision surface construction, which 
can create separation of highly overlapping class populations, uses a dynamic nonlinear 
5 sequence technique (see Griffith, Mathematical Neurobiology , Academic Press, NY, NY, 
Chpt. 8 (1971)) of neural biological origins. This approach is also applicable to the 
application, because it constructs a finite set of random steps in a sequence network of the 
i'^ state, using a step operator S(i). Here, the sequence operator is sampling the 
population density at any level of dimension in subspace N', with the next step sample 

10 address (location vector S„ for | n | = N') derived from the operation of the use of the 
density value as a seed entry into an address table of random values. By using random 
noise dithering, with sampling of neighboring densities, this approach can be shown to 
generate a stable set of addresses after a logarithmic number of iterations (Y) on the 
dimensionality. This is called a terminal cycle (TC) decision mapping and has been 

15 shown to be stable to large dimensionality (N=10^, Y-1253, see the Griffith article). 

An important part of a successful TC approach is to populate the empty N-D 
space grid points with random noise for dithering, such that empty points will still 
generate a different mapping address each time. In the TC approach to decision surfaces, 
the classification structure is contained in the sequence of the address jumps, which can 

20 be thought of as just the link vectors between the object exemplar population. That is to 
say, the address sequence in the TC approach becomes the link-list worm hole density, as 
described in the worldline example. Hence, the inherent ability to represent "knowledge" 
in this N-D space as a set of objects and links through class filtering has been simplified 
to class regional object mapping with NN surfaces contained in the weights of each g^ 

25 hidden layer, and the addresses of the sequence of link vector lists constructs from links 
1^ . In essence, the invention has replaced an object/link density moment model with a 
sequence list of addresses based on class decision surface projections. The final element 
of the invention is the initiation of the N-D space for a given application, and the method 
of auto-construction for timely utilization of the N-D knowledge representation. The 

30 multimodal separation from projected density asynmietry moment expansions is the 



critical beginning point of a priori learning for the NN and uses a gas/liquid dynamic 
model. This model, described next, uses a dynamic stability of continuously changing 
representations. 

5 Initiation and Dynamic Stability 

Referring to Figure 3, the application of the invention is in obtaining a STM 
model for M'- D representation of atomic input/outputs from the RW into an N-D OD 
model. However, it is recognized that the STM will require access to expanding spaces 
as the application changes its RW inputs, and hence the N-D OD model must be 

10 remappable back to LTM storage of M dimensions. This migration of an N-D OD to M- 
D LTM stored in a mass storage media is based on a continuous M'- D RW input/output 
stimulation. This LTM mapping is based on a finite set of class location structures of the 
dynamic address sequence set, and is loaded into/out-of the STM N-D space density 
representation, using predicate calculus location indexing. Hence, the multiple 2-D 

15 structure of the LTM is based on an address sequence for each class (2-D), for each 
subspace representation. The stability and finite time for LTM memory population 
access must critically use a subspace construction in order to limit all fidelity levels of 
searches. Very fast LTM models can be constructed at low level fidelity, and can be 
refined with the addition of expanded dimensions about the initial representation as 

20 described next. 

The synthesis of STM and mapping to LTM involves taxonomy of an isomorphic 
relationship between the N' subspaces to the M < 3 storage space of the LTM media. 
Each subspace dimension has been shown to be mappable down to N' = 2 or 3 spaces, 
and hence has a direct means of storage in LTM. It also has a means of "tagging" the 

25 diversity for each of the SD iterations within this subspace dimension. Hence, the LTM 
construction taxonomy is considered to be a class sequence in N' representations, with a 
large linked list of SD diversities. This allows for expansion of subspace fidelity within a 
single N' representation by adding more "platters" to the LTM. These 2-D physical 
platters form a linked-list construction. Because this is an object class model, the 

30 mapping can be directly controlled by the LTM search algorithms using a predicate 
calculus representation, and avoids any indirect LTM addressing. This is an important 



element of the efficiency for the STM synthesis search as the apphcation requires higher 
fidehty (increasing SD) and higher dimensionahty (increasing N') for maturing or 
stabilizing dynamics of the RW knowledge representation. 

A continuous process in the construction of LTM is the construction of class 

5 decision surfaces in STM that is efficient for a given level of user fidelity. It is the class 
filtering process which creates the HOS fidehty in the subspace projections. Here the use 
of forced choice user decision feedback in an AC are used to quickly identify the 
subspace for the decision surface construction, and the merging or elimination of other 
subspace diversities. A model of the classification dynamics combines two previous 

10 concepts to retain a dynamic classification for those of a) large regional density 
representation, such as a liquid region with a parametric density in p(x^) encompassing 
all pixel grid points, and b) singularly-narrow regional density representations, such as a 
gas region with a discrete set of pixels as a pixel density used to represent p(x^) for RW 
exemplars. This allows for an initiation of the invention by letting N'= M = N for an 

15 initial set of RW objects. Exemplars replace the zeroth state value of random values in 
the STM during initial user interaction. 

The system iterates with N > N', with increasing diversity maintained initially by 
class heuristics being used, either as operator heuristics or user profiles determined from 
the application. As gas regions become more dense and frequently accessed, the modeled 

20 classification migrates to a Hquid model, with a gas model used near the decision 
surfaces. As these regions of liquid modehng become inactive, they migrate back to 
pixels of recent exemplars as a gas model. This continuous action dynamic through RW 
stimulation modifies the STM representation, as well as the LTM migration. The use of 
TC decisions is a means of continuously testing regions of the N-D space for activity or 

25 utility without exhaustive search approaches. This becomes a means of implementing the 
AC formulation for equivocation and uncertainty and hence retains continuous RW user 
feedback to the utility and applicability of the LTM/STM states. The constructed STM 
and LTM representation of user(s) knowledge in this dynamically-classed N-D space 
allows the invention to support a large variety of decision-process applications, where 

30 either a chosen input attribute set is compared as a test to decisively match a class, or an 



object location is projected for an output attribute control to perform a function (e.g. the 
"Grandmother" neuron example used in recognizing a picture of one's grandmother). 

As a further illustration of the invention, the example application of Figure 2 is 
described using the RW interface of a personal PDA hand held computer (a machine, 
5 such as shown in the Figure 1 concept) connected via a wireless communication link to a 
large server via a LAN or the Internet. The PDA displays data output in graphic and 
alphanumeric form, and receives data input in the form of spoken words and "pen" 
scratchings on the display surface, for example, using a typical Palm-Pilot-Iike device. 
One can also add a location RW device for external non-user information, such as a 
10 commercial GPS RF receiver. This constitutes a single application reaUzation of the user 
and sensor interfaces 11, 12 of Figure 3. 

Inside the PDA is an ASIC, which is based in the invention's technology models 
and algorithms, and represents the STM contained in elements 14, 15 and 16 of Figure 3. 
The Core Model 14 of Figure 3 is the essence of the algorithms in the ASIC, with the 
15 interface mapping 16 and projection 15 algorithms utilizing computational storage 
contained in the ASIC and mass storage devices. The link to the mass storage device 13 
is the LTM of Figure 3, and is accessed using simpHstic predicate calculus commands as 
messages over the PDA wireless communication. This mass storage device can be a 
server, which retums RDB types of specific LTM subspace populations for rework by the 
20 STM. Note that here the RDB accessing of LTM avoids previously cited problems in the 
use of SQL for RDB usage. This particular application is shown in Figure 7 for user 
authentication and identification in the e-presence application of Figure 2. 

The utilization of this application device is as follows: 
1) The LTM 31 is used as a remote server through a teleconmiunication interface 
25 Unk (i.e. telecom 36 in Figure 7 to support the STM 32), limited by the ASIC 

resources of the STM realization, through dynamic, but frequent and 

discontinuous access to a RDB storage using a predicate calculus algorithm. 

There, previously captured profiles of the user's behavior derived from 

preferences, correlated events, etc. (e.g. e-presence signature) are stored. 
30 2) The graphic user interface 33 uses a display of characters on the PDA screen in a 

sequential, and rapid manner, with random location and value, and having the 



display time limited to forced-choice cognitive science values (e.g. 5 per second). 
3) The atomic feature input space of M' = 10 is made with commercial voice 
characterization (e.g. LPC) sampled at forced choice response times (e.g. 5 per 
second), and sequential handwritten features based on MV analysis of the pen 

5 scrollings at a similar forced choice sampling (e.g. 5 per second on the temporal 

GUI input sequence). The dimensions of this input representation could be based 
for example on six LPC features for voice input and four MV features for hand 
written input, for a six second session of user ID input verification (37) 
comprising a worldline identity in the N = 10 correlation space. 

10 4) User ID authentication of the e-presence could be expanded to include location- 
dependent ID for further dimensional correlation, and begins with a location- 
dependent (GPS 34) selection (12 of Figure 3) of LTM space exemplars (13 of 
Figure 3). The user is notified graphically (33, 15 of Figure 3) to sign one's name 
with the pen, while simultaneously speaking the same user name. The STM (32, 

15 14 of Figure 3) clocks (35) the sampling of the input data to a set of 30 (6 

sec/200msec) sequential input feature vectors of atomic attributes. These vectors 
construct a linked set of sequential vector subspaces (N' > 10) for a diversity of 
potentially large dimensions (SD > 30). Based on Equation (21), the STM 
representation could be as large as N > 12 for a (N-N)=3 limitation at the atomic 

20 level. However, previous training of the LTM by the user (a few examples), 

would construct class representation of this diversity to potentially only a few 
subspaces, varying with user location. 

5) An adaptive threshold d (set by the designed Pc level) in the test user ID example 
(of name signing with simultaneously speaking the name as speak/sign) is made 

25 in an N = 10 correlation space, which has exponentially increasing reliability to 

simple N = 2 space correlation. In fact, it is the proper sequence and user action, 
which creates a single link of correlation structure of the exemplars as a worldline 
to which the test sample is compared with in LTM training storage, much as is 
done in the TC representation. This feature input concurrency is an important 

30 element of achieving the large Pc values. 

6) Upon "correct" user ID recognition, the PDA further verifies the user ID through 
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a second test, by displaying a sequence of letter and numbers for six seconds in a 
forced choice mode clocking (38), with the user verbally stating each number or 
letter. Again, a subspace of the (N' > 10) atomic input features is used (without 
the pen inputs) for recognition, such as for N' - 6 from before, or also could be 
5 , expanded to N'> 10 if higher fidelity is required. 

This application is a simplistic representation of the STM/LTM partitioning and 
user/sensor interfaces, with the Core Model contained in an ASIC. The ASIC algorithm 
operates using a representation of the subspace atomic inputs, expanded by synthesis to 
create recognition fidelity for class decisions. This expansion adds subspaces and 
10 classes, with dynamic control of recognition within the invention technology application 
product provider, who desires the user ID testing results. This control limits the 
extendibility of the Core Model algorithm in HOS feature order (fourth or more of 
Equation (13) through (17)) and in diversity computation order (m of Equation (13) 
through (17)). 

15 If is limited to a higher dimension through optimum synthesis, then the STM 

represents the recognition correlation very quickly as an n = spatial feature set. Non- 
optimum implementation of the Core Model will result in longer convergence times of n 
< N' and more extensive diversity. 

The invention is applicable to all problems in complexity, such as exemplar 

20 population and attribute normalizations, which require extensions to N >3 spaces for 
representation, and can be shown to be extendable to N= °o without an infinite time 
required for synthesis, due to the linear nature of the TC and worldline approach. This is 
because the representation is limited in fidelity and model hyperspaces with linear growth 
metrics, and hence always deals with a collection of subspace decision surfaces based on 

25 class projections, in order to achieve a higher order correlation analysis. The use of 
heuristics and user profiles allows for the sharing among a set of users of LTM spaces to 
more rapidly initiate the system use, and reduce user training time for forming the 
exemplar population in a minimum time. 

Finally, because of the dynamic user interaction provided by the liquid/gas model 

30 of class decision surfaces, (which allow for the synthesis process to proceed into a rapid 
convergence), the invention can change with all types of variations based on the RW 



environment from the user and sensors of the input, and the control of this environment is 
monitored by the forced-choice sampling of the output effections. 

Example Method and Systems for Applications 

5 An example method is shov^n in a flow chart in Figure 8. The user and/or RW 

sensor inputs begin to stimulate and a response to this stimuli is constructed (40) by first 
capturing this stimuli (40) and storing it, and successive stimuli (41) into the STM 
element (42). Here a synthesis of the OD model construction is performed and the model 
abstraction is stored (43) into LTM (44). During synthesis (42), or as a result of new 

10 stimuh (39), the LTM (44) is used to retrieve the synthesized model (45) and either a new 
model is synthesized as a dynamic to further stimuh (42, 39, 40) or an output response is 
generated (46) and drives the user and/or RW effector outputs (47). The dynamic 
synthesis can also request new user and/or RW input (39) as requested by the synthesis 
(46) of the STM (42). Specific functions of this method embodied in the text are hsted in 

15 Figure 8. 

An example system which creates the implementation of this method of Figure 8 
is shown in Figure 9 as a generic block diagram. The user and/or RW input interface 48 
is based on commercial standards and components, and interfaces to a data capture device 
49 embodied as either a process or specialized board, an ASIC, or a net-list which 

20 provides the STM component 50 with input data as a stimuli. The STM utilizes a 
floating-point processor component with memory to scale the input data values to fixed 
point storage for use in the model synthesis process. The model synthesis occurs in the 
STM components 50 which execute fixed point processors and data value arithmetic 
operations, driven by a processor and memory. The STM controls this synthesis by using 

25 a control processor operating on the synthesis parameters and creating testing decision 
outcomes that are stored in memory. This control processor uses a synthesis processor 
code component that orchestrates the synthesis process and stores and retrieves results 
with a mass storage LTM device through a processor, memory, and communication 
device 54, with the LTM becoming as an example, a commercial RDB system 5L The 

30 results of the dynamic model synthesis and control process, based on the input stimuli 48 
are outputted through a data generator device 52 which conveys output data for display 



for the user and/or used by the RW output effectors 53. These generic components of 
Figure 9 can be embodied in a system of commercial devices and components, a 
processor executing software with external input/output devices and mass storage 
devices, an ASIC device designed with components of input/output interfaces to other 
5 commercial suppliers, or as a net-list for use by other commercial suppliers which is 
incorporated into specific application hardware product systems. 

Each of the documents listed above is incorporated herein by reference. 

While particular embodiments of the present invention have been described and 
illustrated, it should be understood that the invention is not limited thereto because 
10 modifications may be made by persons skilled in the art. The present application 
contemplates any and all modifications that fall within the spirit and scope of the 
underlying invention discloses and claimed herein. 



